Stereo Matching for Unconstrained Face Recognition - Ph.D. Proposal

Unconstrained face recognition is a problem of fundamental importance in computer vision. We propose to address this problem by using stereo matching to judge the similarity of two, 2D images of faces. Stereo matching allows for arbitrary, physically valid, continuous correspondences. We show that the stereo matching cost provides a very robust measure of similarity of faces that is insensitive to a wide range of variations. To enable this, we show that for conditions common in face recognition, the epipolar geometry of face images can be computed using either four or three feature points. We also provide a straightforward adaptation of a stereo matching algorithm to compute the similarity between faces. The proposed approach has been tested on the CMU PIE dataset and demonstrates superior performance compared to existing methods in the presence of pose variation. We have also studied the problem of face recognition with weight variation. Using this dataset, we empirically study how weight gain and loss affects face recognition performance. We present baseline experiments using a wide variety of existing methods. These show that weight change can significantly degrade the performance of recognition algorithms. Our results also show that correspondence-based methods exhibit the most robust performance as the weight difference increases. The research plans include: building a stereo method for matching in the presence of deformation and illumination change and learning from pose invariant descriptors built using cost and correspondence information.